{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "blogpost: true\n",
    "date: 29 Mar 2024\n",
    "author: Douglas Jia\n",
    "tags: Profiling, HIP, ROCm, Tracing, HPC \n",
    "category: Software Tools and Optimization\n",
    "language: English\n",
    "---\n",
    "<head>\n",
    "  <meta charset=\"UTF-8\">\n",
    "  <meta name=\"description\" content=\"Unveiling Performance Insights: Profiling and Tracing Applications on AMD GPU with rocmProfileData\">\n",
    "  <meta name=\"keywords\" content=\"Profiling, Tracing, HPC, HIP, rocmProfileData, AMD, GPU, MI300, MI250, MI210, rocmProfileData\">\n",
    "</head>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Unveiling Performance Insights: Profiling and Tracing Applications on AMD GPU with rocmProfileData\n",
    "\n",
    "In this blog, we delve into the capabilities of rocmProfileData, a powerful tool developed by AMD for profiling and tracing applications across various programming languages on AMD GPU. Our goal is to equip developers and data scientists with the insights and resources to fully leverage the performance potential of their GPU-accelerated applications in production, using a straightforward code example.\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In the dynamic landscape of GPU-accelerated computing, achieving optimal performance and efficiency is key to improve user experience and increase revenue. This pursuit often leads engineers and data scientists to delve into the realms of profiling and tracing, two indispensable techniques for gaining deep insights into the behavior and performance characteristics of applications running on GPUs.\n",
    "\n",
    "Profiling focuses on analyzing performance metrics to quantify the behavior of the application during execution. By profiling key performance indicators such as execution time, memory usage, and kernel occupancy, developers can pinpoint areas of inefficiency and prioritize optimization efforts. Profiling provides actionable insights into the runtime behavior of the application, guiding developers towards optimizations that yield tangible performance improvements.\n",
    "\n",
    "Tracing, on the other hand, involves monitoring and recording the sequence of operations performed by an application as it executes on the GPU. This detailed log provides invaluable visibility into the inner workings of the application, allowing developers to understand how data flows through the computation pipeline, identify potential bottlenecks, and optimize algorithmic implementations. Tracing essentially offers a \"birds-eye view\" of the application's execution, enabling developers to diagnose performance issues and fine-tune their code for maximum efficiency.\n",
    "\n",
    "The importance of profiling and tracing cannot be overstated in the context of GPU-accelerated computing. As GPUs continue to play a pivotal role in a wide range of applications, specifically generative AI applications serving large models, understanding and optimizing their performance can generate more revenue by improving end user experience and lowering model serving cost. Profiling and tracing empower developers to unlock the full potential of GPU hardware, enabling them to harness its parallel computing power efficiently and effectively.\n",
    "\n",
    "Developed in-house at AMD by Michael Wootton and his colleagues, [rocmProfileData](https://github.com/ROCm/rocmProfileData) is specifically tailored to profile and trace applications running with ROCm on AMD GPUs. I am grateful for Michael's assistance in writing this blog and for providing invaluable documentation for reference. Throughout this blog, we will demonstrate how easily the core functionalities of this package can be executed with the help of an informative code example.\n",
    "\n",
    "## Environment setup\n",
    "\n",
    "We run the code example in a PyTorch ROCm 6.02 docker container (for a list of supported OS and hardware by AMD, please [click here](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html)) with an AMD GPU in Ubuntu.\n",
    "\n",
    "Pull and run the docker container with the code below in a Linux shell:\n",
    "\n",
    "```text\n",
    "docker run -it --ipc=host --network=host --device=/dev/kfd --device=/dev/dri \\\n",
    "           --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined \\\n",
    "           --name=starcoder rocm/pytorch:rocm6.02_ubuntu22.04_py3.10_pytorch_2.1.2 /bin/bash\n",
    "```\n",
    "\n",
    "You can verify the number of GPUs detected by PyTorch on your machine by executing the following two lines of code in the Python console. If the Docker configuration is correct, the detected number of GPUs should match the number of GPUs installed on your machine.\n",
    "\n",
    "```python\n",
    "import torch\n",
    "torch.cuda.device_count()\n",
    "```\n",
    "\n",
    "## Implementation\n",
    "\n",
    "### Package installation\n",
    "\n",
    "Install rocmProfileData and other required softwares by running the following commands in your Linux Shell.\n",
    "\n",
    "```text\n",
    "apt-get install libfmt-dev\n",
    "\n",
    "git clone https://github.com/ROCm/rocmProfileData.git\n",
    "cd rocmProfileData\n",
    "make; make install\n",
    "cd ..\n",
    "```\n",
    "\n",
    "### Profiling a PyTorch multiplication function\n",
    "\n",
    "In this code example, we'll profile a Python script featuring matrix multiplication implemented using PyTorch. You can locate the script `matrix_mult.py` in the `src` folder of this blog's GitHub repository: [Link](https://github.com/ROCm/rocm-blogs/tree/release/blogs/artificial-intelligence/rocm-profile-data). Alternatively, you can create your own `matrix_mult.py` file by copying the code provided below.\n",
    "\n",
    "```python\n",
    "import argparse\n",
    "import torch\n",
    "\n",
    "def matmult_gpu(input_data, weights):\n",
    "    \"\"\"\n",
    "    Perform matrix multiplication of two tensors on GPU.\n",
    "    \n",
    "    Args:\n",
    "    input_data (torch.Tensor): Input tensor.\n",
    "    weights (torch.Tensor): Weight tensor.\n",
    "    \n",
    "    Returns:\n",
    "    torch.Tensor: Result of matrix multiplication.\n",
    "    \"\"\"\n",
    "    # Creating tensors on GPU\n",
    "    input_data = input_data.to('cuda')\n",
    "    weights = weights.to('cuda')\n",
    "    \n",
    "    # Optimized matrix multiplication using torch.matmul\n",
    "    output = torch.matmul(input_data, weights)\n",
    "    \n",
    "    return output\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    parser = argparse.ArgumentParser(description='Perform matrix multiplication of two tensors.')\n",
    "    parser.add_argument('--x_shape', nargs=2, type=int, default=[1000, 500], metavar=('N', 'M'), help='Shape of input data matrix')\n",
    "    parser.add_argument('--w_shape', nargs=2, type=int, default=[500, 500], metavar=('J', 'K'), help='Shape of weight matrix')\n",
    "    args = parser.parse_args()\n",
    "\n",
    "    input_data = torch.randn(*args.x_shape)\n",
    "    weights = torch.randn(*args.w_shape)\n",
    "\n",
    "    output = matmult_gpu(input_data, weights)    \n",
    "    print(f'Shape of input data matrix: {args.x_shape}, weight matrix: {args.w_shape}, result matrix:{output.shape}')\n",
    "    print(output)\n",
    "```\n",
    "\n",
    "The primary command for rocmProfileData is runTracer.sh. To profile a Python script, execute the following command: runTracer.sh <-o filename.rpd> python python_script.py <arguments_to_the_python_script>, where the optional -o filename.rpd flag specifies the output file name. The output file, formatted as .rpd, can be queried using SQLite3. In our case, to profile the function specified in matrix_mult.py, we'll execute the following command:\n",
    "\n",
    "```text\n",
    "runTracer.sh -o matmul_result.rpd python matrix_mult.py --x_shape 50000 10000 --w_shape 10000 800\n",
    "```   \n",
    "\n",
    "The output file matmul_result.rpd should be located in the directory where you executed the above command. Now, let's explore the output! You can directly query the table in the shell by running sqlite3 matmul_result.rpd, or in Python using the sqlite3 class. In this blog, we'll utilize the latter approach.\n",
    "\n",
    "Before executing any Python code, ensure you have installed pandas in the shell:\n",
    "\n",
    "```text\n",
    "pip install pandas\n",
    "```\n",
    "\n",
    "Then, import the required packages in Python:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before delving into the profiling metrics, let's first examine all the tables and views in the .rpd file and understand how the schemas are defined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>type</th>\n",
       "      <th>schema</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>rocpd_kernelcodeobject</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_kernelcodeobject\" (\"id\" in...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>sqlite_sequence</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE sqlite_sequence(name,seq)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_string</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_string\" (\"id\" integer NOT ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_barrierop</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_barrierop\" (\"op_ptr_id\" in...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_copyapi</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_copyapi\" (\"api_ptr_id\" int...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_op_inputSignals</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_op_inputSignals\" (\"id\" int...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_op</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_op\" (\"id\" integer NOT NULL...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_api</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_api\" (\"id\" integer NOT NUL...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_api_ops</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_api_ops\" (\"id\" integer NOT...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_kernelapi</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_kernelapi\" (\"api_ptr_id\" i...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_metadata</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_metadata\" (\"id\" integer NO...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>rocpd_monitor</th>\n",
       "      <td>table</td>\n",
       "      <td>CREATE TABLE \"rocpd_monitor\" (\"id\" integer NOT...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>api</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW api AS SELECT rocpd_api.id,pid,tid...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>op</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW op AS SELECT rocpd_op.id,gpuId,que...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>busy</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW busy AS select A.gpuId, GpuTime, W...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ktop</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW ktop as select C.string as Name, c...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW top as select C.string as Name, co...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>kernel</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW kernel AS SELECT B.id, gpuId, queu...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>copy</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW copy AS SELECT B.id, pid, tid, sta...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>copyop</th>\n",
       "      <td>view</td>\n",
       "      <td>CREATE VIEW copyop AS SELECT B.id, gpuId, queu...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         type  \\\n",
       "name                            \n",
       "rocpd_kernelcodeobject  table   \n",
       "sqlite_sequence         table   \n",
       "rocpd_string            table   \n",
       "rocpd_barrierop         table   \n",
       "rocpd_copyapi           table   \n",
       "rocpd_op_inputSignals   table   \n",
       "rocpd_op                table   \n",
       "rocpd_api               table   \n",
       "rocpd_api_ops           table   \n",
       "rocpd_kernelapi         table   \n",
       "rocpd_metadata          table   \n",
       "rocpd_monitor           table   \n",
       "api                      view   \n",
       "op                       view   \n",
       "busy                     view   \n",
       "ktop                     view   \n",
       "top                      view   \n",
       "kernel                   view   \n",
       "copy                     view   \n",
       "copyop                   view   \n",
       "\n",
       "                                                                   schema  \n",
       "name                                                                       \n",
       "rocpd_kernelcodeobject  CREATE TABLE \"rocpd_kernelcodeobject\" (\"id\" in...  \n",
       "sqlite_sequence                    CREATE TABLE sqlite_sequence(name,seq)  \n",
       "rocpd_string            CREATE TABLE \"rocpd_string\" (\"id\" integer NOT ...  \n",
       "rocpd_barrierop         CREATE TABLE \"rocpd_barrierop\" (\"op_ptr_id\" in...  \n",
       "rocpd_copyapi           CREATE TABLE \"rocpd_copyapi\" (\"api_ptr_id\" int...  \n",
       "rocpd_op_inputSignals   CREATE TABLE \"rocpd_op_inputSignals\" (\"id\" int...  \n",
       "rocpd_op                CREATE TABLE \"rocpd_op\" (\"id\" integer NOT NULL...  \n",
       "rocpd_api               CREATE TABLE \"rocpd_api\" (\"id\" integer NOT NUL...  \n",
       "rocpd_api_ops           CREATE TABLE \"rocpd_api_ops\" (\"id\" integer NOT...  \n",
       "rocpd_kernelapi         CREATE TABLE \"rocpd_kernelapi\" (\"api_ptr_id\" i...  \n",
       "rocpd_metadata          CREATE TABLE \"rocpd_metadata\" (\"id\" integer NO...  \n",
       "rocpd_monitor           CREATE TABLE \"rocpd_monitor\" (\"id\" integer NOT...  \n",
       "api                     CREATE VIEW api AS SELECT rocpd_api.id,pid,tid...  \n",
       "op                      CREATE VIEW op AS SELECT rocpd_op.id,gpuId,que...  \n",
       "busy                    CREATE VIEW busy AS select A.gpuId, GpuTime, W...  \n",
       "ktop                    CREATE VIEW ktop as select C.string as Name, c...  \n",
       "top                     CREATE VIEW top as select C.string as Name, co...  \n",
       "kernel                  CREATE VIEW kernel AS SELECT B.id, gpuId, queu...  \n",
       "copy                    CREATE VIEW copy AS SELECT B.id, pid, tid, sta...  \n",
       "copyop                  CREATE VIEW copyop AS SELECT B.id, gpuId, queu...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conn = sqlite3.connect('matmul_result.rpd')\n",
    "# Execute SQL query to get the table names\n",
    "tables = conn.execute(\"SELECT name, type, sql FROM sqlite_master where type='table' or type='view';\").fetchall()\n",
    "table_view = pd.DataFrame(data=tables, columns=['name', 'type', 'schema']).set_index('name')\n",
    "conn.close()\n",
    "table_view"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find the queries that defined these tables/views in the `schema` column. Below, we'll introduce several key tables/views essential for profiling your application:\n",
    "\n",
    "- **rocpd_op**: GPU operations are stored in the `rocpd_op` table, serving as a base class for GPU operations. Additional \"subclass tables\" exist to accommodate extra details for specific operation types, such as size for Copy operations and grid size for Kernel operations. Entries in these subclass tables reference the base operation entry for common information, such as GPU, stream, begin, and end.\n",
    "\n",
    "- **rocpd_api**: CPU-based calls are recorded in the `rocpd_api` table, typically comprising HIP API calls and ROCTX marks/ranges, among other entries. For instance, PyTorch's internal profiler emits time ranges for its internal operators, facilitating the analysis of interactions between PyTorch operators and HIP.\n",
    "\n",
    "- **rocpd_kernelapi**: API calls launching kernels can store additional parameters in the `rocpd_kernelapi` table. Rows in this table reference entries in the `rocpd_api` table for base class fields and can be joined using `rocpd_kernelapi.api_ptr_id = rocpd_api.id`.\n",
    "\n",
    "- **rocpd_copyapi**: API calls performing copies can store extra parameters in the `rocpd_copyapi` table. Similar to `rocpd_kernelapi`, entries in this table reference entries in the `rocpd_api` table for base class fields and can be joined using `rocpd_copyapi.api_ptr_id = rocpd_api.id`.\n",
    "\n",
    "- **api**: The `rocpd_api` table with expanded strings for the 'apiName' and 'args' columns.\n",
    "\n",
    "- **op**: The `rocpd_op` table with expanded strings for the 'description' and 'opType' columns.\n",
    "\n",
    "- **busy**: Displays the percentage of GPU utilization for each GPU, averaged over the entire trace. For accuracy, the trace should not include \"warmup\" or should be sufficiently long.\n",
    "\n",
    "- **top**: Presents a list of operations consuming the most GPU time.\n",
    "\n",
    "- **ktop**: Lists kernel operations consuming the most time, excluding async copies and barriers.\n",
    "\n",
    "- **kernel**: Displays kernel launch parameters for each kernel.\n",
    "\n",
    "- **copy**: Shows all copy API calls with their parameters, including CPU timestamps.\n",
    "\n",
    "- **copyop**: Presents copy API calls and parameters for asynchronous copies, i.e., copies resulting in GPU operations. Includes GPU timestamps and represents a subset of all copies.\n",
    "\n",
    "Execute the following code block to load the tables/views into pandas dataframes. You can revise the code to load and explore other tables/views you are interested in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "conn = sqlite3.connect(\"matmul_result.rpd\")\n",
    "df_op = pd.read_sql_query(\"SELECT * from op\", conn)\n",
    "df_top = pd.read_sql_query(\"SELECT * from top\", conn)\n",
    "df_ktop = pd.read_sql_query(\"SELECT * from ktop\", conn)\n",
    "df_busy = pd.read_sql_query(\"SELECT * from busy\", conn)\n",
    "conn.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>gpuId</th>\n",
       "      <th>queueId</th>\n",
       "      <th>sequenceId</th>\n",
       "      <th>start</th>\n",
       "      <th>end</th>\n",
       "      <th>description</th>\n",
       "      <th>opType</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8306017670485923</td>\n",
       "      <td>8306017825375030</td>\n",
       "      <td></td>\n",
       "      <td>CopyHostToDevice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8306017826047669</td>\n",
       "      <td>8306017828336625</td>\n",
       "      <td></td>\n",
       "      <td>CopyHostToDevice</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8306020565922417</td>\n",
       "      <td>8306020592134373</td>\n",
       "      <td>Cijk_Ailk_Bljk_SB_MT128x64x16_MI32x32x2x1_SN_1...</td>\n",
       "      <td>KernelExecution</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8306020592134373</td>\n",
       "      <td>8306020592140613</td>\n",
       "      <td>void at::native::(anonymous namespace)::CatArr...</td>\n",
       "      <td>KernelExecution</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8306020592140613</td>\n",
       "      <td>8306020592146693</td>\n",
       "      <td>void at::native::(anonymous namespace)::CatArr...</td>\n",
       "      <td>KernelExecution</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   id  gpuId  queueId  sequenceId             start               end  \\\n",
       "0   1      2        0           0  8306017670485923  8306017825375030   \n",
       "1   2      2        0           0  8306017826047669  8306017828336625   \n",
       "2   3      2        0           0  8306020565922417  8306020592134373   \n",
       "3   4      2        0           0  8306020592134373  8306020592140613   \n",
       "4   5      2        0           0  8306020592140613  8306020592146693   \n",
       "\n",
       "                                         description            opType  \n",
       "0                                                     CopyHostToDevice  \n",
       "1                                                     CopyHostToDevice  \n",
       "2  Cijk_Ailk_Bljk_SB_MT128x64x16_MI32x32x2x1_SN_1...   KernelExecution  \n",
       "3  void at::native::(anonymous namespace)::CatArr...   KernelExecution  \n",
       "4  void at::native::(anonymous namespace)::CatArr...   KernelExecution  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_op.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TotalCalls</th>\n",
       "      <th>TotalDuration</th>\n",
       "      <th>Ave</th>\n",
       "      <th>Percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CopyHostToDevice</td>\n",
       "      <td>2</td>\n",
       "      <td>157178</td>\n",
       "      <td>78589</td>\n",
       "      <td>85.460920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cijk_Ailk_Bljk_SB_MT128x64x16_MI32x32x2x1_SN_1...</td>\n",
       "      <td>1</td>\n",
       "      <td>26211</td>\n",
       "      <td>26211</td>\n",
       "      <td>14.251975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>CopyDeviceToHost</td>\n",
       "      <td>53</td>\n",
       "      <td>334</td>\n",
       "      <td>6</td>\n",
       "      <td>0.182015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>void at::native::(anonymous namespace)::CatArr...</td>\n",
       "      <td>6</td>\n",
       "      <td>37</td>\n",
       "      <td>6</td>\n",
       "      <td>0.020182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>void at::native::reduce_kernel&lt;512, 1, at::nat...</td>\n",
       "      <td>1</td>\n",
       "      <td>19</td>\n",
       "      <td>19</td>\n",
       "      <td>0.010787</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name  TotalCalls  \\\n",
       "0                                   CopyHostToDevice           2   \n",
       "1  Cijk_Ailk_Bljk_SB_MT128x64x16_MI32x32x2x1_SN_1...           1   \n",
       "2                                   CopyDeviceToHost          53   \n",
       "3  void at::native::(anonymous namespace)::CatArr...           6   \n",
       "4  void at::native::reduce_kernel<512, 1, at::nat...           1   \n",
       "\n",
       "   TotalDuration    Ave  Percentage  \n",
       "0         157178  78589   85.460920  \n",
       "1          26211  26211   14.251975  \n",
       "2            334      6    0.182015  \n",
       "3             37      6    0.020182  \n",
       "4             19     19    0.010787  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_top.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TotalCalls</th>\n",
       "      <th>TotalDuration</th>\n",
       "      <th>Ave</th>\n",
       "      <th>Percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>void at::native::index_elementwise_kernel&lt;128,...</td>\n",
       "      <td>1</td>\n",
       "      <td>9410</td>\n",
       "      <td>9410</td>\n",
       "      <td>14.442580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>void at::native::(anonymous namespace)::CatArr...</td>\n",
       "      <td>6</td>\n",
       "      <td>8390</td>\n",
       "      <td>1398</td>\n",
       "      <td>12.877698</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>void at::native::reduce_kernel&lt;512, 1, at::nat...</td>\n",
       "      <td>1</td>\n",
       "      <td>5417</td>\n",
       "      <td>5417</td>\n",
       "      <td>8.315085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>void at::native::modern::elementwise_kernel&lt;at...</td>\n",
       "      <td>1</td>\n",
       "      <td>5334</td>\n",
       "      <td>5334</td>\n",
       "      <td>8.187123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>void at::native::modern::elementwise_kernel&lt;at...</td>\n",
       "      <td>2</td>\n",
       "      <td>5122</td>\n",
       "      <td>2561</td>\n",
       "      <td>7.861807</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name  TotalCalls  \\\n",
       "0  void at::native::index_elementwise_kernel<128,...           1   \n",
       "1  void at::native::(anonymous namespace)::CatArr...           6   \n",
       "2  void at::native::reduce_kernel<512, 1, at::nat...           1   \n",
       "3  void at::native::modern::elementwise_kernel<at...           1   \n",
       "4  void at::native::modern::elementwise_kernel<at...           2   \n",
       "\n",
       "   TotalDuration   Ave  Percentage  \n",
       "0           9410  9410   14.442580  \n",
       "1           8390  1398   12.877698  \n",
       "2           5417  5417    8.315085  \n",
       "3           5334  5334    8.187123  \n",
       "4           5122  2561    7.861807  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_ktop.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gpuId</th>\n",
       "      <th>GpuTime</th>\n",
       "      <th>WallTime</th>\n",
       "      <th>Busy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>183918056</td>\n",
       "      <td>2964070817</td>\n",
       "      <td>0.062049</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   gpuId    GpuTime    WallTime      Busy\n",
       "0      2  183918056  2964070817  0.062049"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_busy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upon reviewing the `df_top` and `df_busy` metrics, you may observe that the actual computation time on the GPU is significantly shorter compared to the total running time, with the actual matrix computation time being even shorter. This discrepancy is primarily attributed to the substantial overhead incurred by data movement between different hardware components. Consequently, there are instances where applications running solely on the CPU may outperform those utilizing the GPU, as they eliminate the need for data transfer between CPU and GPU. However, as the size of the data increases, this overhead becomes proportionally smaller, highlighting the superior performance capabilities of the GPU. To further illustrate this point, let's profile the same operation with much larger data and reexamine the metrics:\n",
    "\n",
    "```text\n",
    "runTracer.sh -o matmul_result_large.rpd python matrix_mult.py --x_shape 100000 50000 --w_shape 50000 800\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Name</th>\n",
       "      <th>TotalCalls</th>\n",
       "      <th>TotalDuration</th>\n",
       "      <th>Ave</th>\n",
       "      <th>Percentage</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CopyHostToDevice</td>\n",
       "      <td>2</td>\n",
       "      <td>1577218</td>\n",
       "      <td>788609</td>\n",
       "      <td>74.727344</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Cijk_Ailk_Bljk_SB_MT64x64x16_MI32x32x2x1_SN_1L...</td>\n",
       "      <td>1</td>\n",
       "      <td>532802</td>\n",
       "      <td>532802</td>\n",
       "      <td>25.243742</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>CopyDeviceToHost</td>\n",
       "      <td>53</td>\n",
       "      <td>420</td>\n",
       "      <td>7</td>\n",
       "      <td>0.019931</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>void at::native::(anonymous namespace)::CatArr...</td>\n",
       "      <td>6</td>\n",
       "      <td>34</td>\n",
       "      <td>5</td>\n",
       "      <td>0.001637</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>void at::native::reduce_kernel&lt;512, 1, at::nat...</td>\n",
       "      <td>1</td>\n",
       "      <td>20</td>\n",
       "      <td>20</td>\n",
       "      <td>0.000963</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                Name  TotalCalls  \\\n",
       "0                                   CopyHostToDevice           2   \n",
       "1  Cijk_Ailk_Bljk_SB_MT64x64x16_MI32x32x2x1_SN_1L...           1   \n",
       "2                                   CopyDeviceToHost          53   \n",
       "3  void at::native::(anonymous namespace)::CatArr...           6   \n",
       "4  void at::native::reduce_kernel<512, 1, at::nat...           1   \n",
       "\n",
       "   TotalDuration     Ave  Percentage  \n",
       "0        1577218  788609   74.727344  \n",
       "1         532802  532802   25.243742  \n",
       "2            420       7    0.019931  \n",
       "3             34       5    0.001637  \n",
       "4             20      20    0.000963  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "conn = sqlite3.connect(\"matmul_result_large.rpd\")\n",
    "df_top = pd.read_sql_query(\"SELECT * from top\", conn)\n",
    "df_busy = pd.read_sql_query(\"SELECT * from busy\", conn)\n",
    "conn.close()\n",
    "df_top.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gpuId</th>\n",
       "      <th>GpuTime</th>\n",
       "      <th>WallTime</th>\n",
       "      <th>Busy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>2110631504</td>\n",
       "      <td>4933119382</td>\n",
       "      <td>0.427849</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   gpuId     GpuTime    WallTime      Busy\n",
       "0      2  2110631504  4933119382  0.427849"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_busy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, take note that the matrix multiplication kernel running time on the GPU has risen from 14.25% to 25.24%, and the total GPU time has increased from 6.2% to 42.78%. In summary, these tables provide the necessary metrics for profiling your application.\n",
    "\n",
    "### How to trace an application\n",
    "\n",
    "You might find it surprising that we can trace an application using the `.rpd` file, as it already contains all the necessary information for tracing an application. All you need to do is convert the `.rpd` file to a `.json` file using the following command, allowing it to be imported into trace viewers like [Chrome Trace](chrome://tracing). Please ensure to adjust the path to the `rpd2tracing.py` script for it to run properly. Once the command completes, you'll find the output `matmul_result.json` file in your working directory.\n",
    "\n",
    "```text\n",
    "python3 ../rocmProfileData/tools/rpd2tracing.py matmul_result.rpd matmul_result.json\n",
    "```\n",
    "\n",
    "Now, you can download the `matmul_result.json` file and import it into [Chrome Trace](chrome://tracing) to explore the traces. Below is a snippet of the trace displaying the matrix multiplication kernel.\n",
    "\n",
    "![](image/matmul_trace.png)\n",
    "\n",
    "In this blog, we've introduced the primary functionalities of rocmProfileData. Feel free to explore additional useful features and functionalities by visiting its [Github page](https://github.com/ROCm/rocmProfileData/blob/master/FEATURES.md).\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py_3.10",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
